Skip to content

feat(backends): add Whisper-Medusa transcription - #11290

Open
localai-org-maint-bot wants to merge 2 commits into
masterfrom
bot/issue-3127-whisper-medusa
Open

feat(backends): add Whisper-Medusa transcription#11290
localai-org-maint-bot wants to merge 2 commits into
masterfrom
bot/issue-3127-whisper-medusa

Conversation

@localai-org-maint-bot

Copy link
Copy Markdown
Collaborator

What does this PR do?

Adds a dedicated Python gRPC backend for aiola Whisper-Medusa checkpoints.

  • loads local paths or Hugging Face model identifiers through the standard model resolver
  • normalizes input to mono 16 kHz audio and enforces upstream’s 30-second clip limit
  • exposes language and generation regulation options
  • adds Linux amd64 CPU and NVIDIA CUDA 12 backend images
  • registers the backend in the backend gallery and Make targets
  • documents configuration, supported checkpoints, and upstream limitations

The upstream project is archived and pins PyTorch 2.2.2. This PR intentionally does not advertise macOS, ROCm, Intel GPU, or Jetson images because upstream has not published compatibility for those targets.

Closes #3127

Testing

  • python3 -m unittest discover -s backend/python/whisper-medusa -p "test_unit.py"
  • python3 -m py_compile backend/python/whisper-medusa/backend.py backend/python/whisper-medusa/test_unit.py
  • git diff --check
  • YAML parse of .github/backend-matrix.yml and backend/index.yaml with gopkg.in/yaml.v3
  • make -n docker-build-whisper-medusa BUILD_TYPE= BASE_IMAGE=ubuntu:24.04

The focused Go gallery suite could not run because generated protobuf stubs are absent in a fresh worktree and the pinned protoc download was rejected by the environment proxy with HTTP 403. A full backend image build was not run because it downloads the multi-gigabyte Whisper-Medusa model/dependency stack.

Checklist

  • Tests added for new backend helpers
  • Backend build matrix and gallery metadata updated
  • User-facing documentation added
  • No coverage baseline or tolerance changes

Add a dedicated Python gRPC backend for aiola Whisper-Medusa checkpoints, including mono 16 kHz preprocessing, bounded clip validation, CPU/CUDA 12 images, backend gallery metadata, and user documentation.

Assisted-by: Codex:gpt-5
The standalone Python release used by libbackend does not publish a 3.11.18 artifact. Pin Whisper-Medusa to the available 3.11.13 build and cover the generated download URL.

Assisted-by: Codex:gpt-5 [systematic-debugging]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

new engine: add whisper-medusa support

1 participant